Papers with distributional models
Formal Semantic Controls over Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Text embeddings provide a concise representation of the semantics of sentences and larger spans of text, rather than individual words, capturing a wide range of linguistic features. |
| Approach: | They propose to shorten the gap between latent semantics and formal symbolics by comparing distributional models to symbolic models grounded on formal linguistics and well-defined mathematical properties. |
| Outcome: | This paper examines the analysis and control of text representations, covering methods from pooling to LLM-based. |
Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences (2022.starsem-1)
Copied to clipboard
| Challenge: | a rise in performance in NLP has led to a decrease in interpretability . a recent study examined how neural semantic models capture relational knowledge . |
| Approach: | They evaluate how well English and Spanish semantic spaces capture a particular type of relational knowledge . they also explore the role of co-occurrences in this context . |
| Outcome: | The proposed model can be used to predict traits associated with concepts in English and Spanish. |
Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text (D19-60)
Copied to clipboard
| Challenge: | Existing work on modeling semantic plausibility has focused on physical plausability but distributional methods fail when tested in supervised settings. |
| Approach: | They propose to use large pretrained language models to model plausibility in supervised settings by extracting attested events from a large corpus and injecting explicit commonsense knowledge into a distributional model. |
| Outcome: | The proposed model is effective in modeling plausibility in a supervised setting. |
Comparing Probabilistic, Distributional and Transformer-Based Models on Logical Metonymy Interpretation (2020.aacl-main)
Copied to clipboard
| Challenge: | Logical metonymies are type clashes between an event-selecting verb and an entity-denoting noun . they are typically interpreted by inferring a hidden event on the basis of contextual cues . |
| Approach: | They propose to use probabilistic and distributional models to model logical metonymy interpretation . they compare models with the best Transformer-based models and some traditional distributional ones . |
| Outcome: | The proposed models perform well on a complex scenario, but low performance on some datasets suggests that logical metonymy is still a challenging phenomenon for computational modeling. |
A Tale of Two Laws of Semantic Change: Predicting Synonym Changes with Distributional Semantic Models (2023.starsem-1)
Copied to clipboard
| Challenge: | Lexical Semantic Change is the study of how the meaning of words evolves through time. |
| Approach: | They propose to use distributional models to detect whether LD or LPC operate for given word pairs. |
| Outcome: | The proposed frameworks achieve a balanced accuracy above 0.6 on the dataset. |
A Retrofitting Model for Incorporating Semantic Relations into Word Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | Existing word embedding models mix semantic similarity with other types of relatedness. |
| Approach: | They propose a model that leverages relational knowledge available in a knowledge resource to improve word embeddings. |
| Outcome: | The proposed model improves word embeddings on synonymy, antonymy and hypernymy relations in WordNet and significantly improves lexical entailment detection task. |
Modeling Event Plausibility with Consistent Conceptual Abstraction (2021.naacl-main)
Copied to clipboard
| Challenge: | Understanding natural language requires common sense, one aspect of which is the ability to discern the plausibility of events. |
| Approach: | They propose a method of forcing model consistency that improves correlation with human plausibility judgements. |
| Outcome: | The proposed method improves correlation with human plausibility judgements. |
BiRRE: Learning Bidirectional Residual Relation Embeddings for Supervised Hypernymy Detection (2020.acl-main)
Copied to clipboard
| Challenge: | supervised hypernymy detection has been studied under various frameworks . supervised classifiers are more likely to suffer from "lexical memorization" |
| Approach: | They propose a representation learning framework called Bidirectional Residual Relation Embeddings to model the possibility of a term being mapped to another in the embedding space by hypernymy relations. |
| Outcome: | The proposed model outperforms baselines over evaluation frameworks. |
Montague semantics and modifier consistency measurement in neural language models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on distributional language models have been focused on linguistics and their relationship with semantic formalisms for decades. |
| Approach: | They propose a method for measuring compositional behavior in contemporary language embedding models by introducing three new tests inspired by Montague semantics. |
| Outcome: | The proposed method measures compositional behavior in language embedding models on adjectival modifier phenomena in adjective-noun phrases. |
Syn2Vec: Synset Colexification Graphs for Lexical Semantic Similarity (2022.naacl-main)
Copied to clipboard
| Challenge: | In this paper we examine patterns of colexification as an aspect of lexical-semantic organization, and compare several approaches to build large scale graphs across 499 world languages. |
| Approach: | They propose to use patterns of colexification as an aspect of lexical-semantic organization to build large scale synset graphs across a typologically diverse set of 499 world languages. |
| Outcome: | The proposed models are evaluated against human judgments on a semantic similarity task for nine languages. |
Distributional Term Set Expansion (L18-1)
Copied to clipboard
| Challenge: | Iterative term set expansion methods for distributional semantic models are used to label terms belonging to a sought after term set. |
| Approach: | They compare iterative term set expansion methods for distributional semantic models to the Simple Margin method, an active learning approach to classification using Support Vector Machines. |
| Outcome: | The proposed methods outperform centrality and classification based methods for distributional semantic models over five different term sets. |
The Contextual Variability of English Nouns: The Impact of Categorical Specificity beyond Conceptual Concreteness (2024.lrec-main)
Copied to clipboard
| Challenge: | Empirical studies on conceptual abstraction have examined differences in contextual distributions of abstract and concrete concept words. |
| Approach: | They propose to use a model to investigate the interplay between contextual variability and specificity of abstract and concrete concepts. |
| Outcome: | The proposed models show that more specific words have closer contexts than generic terms. |